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Method and Apparatus for Image Acquisition, Organization, 
Manipulation, and Publication 

FIELD OF THE INVENTION 

5 

The present invention relates to the field of digital photography. In 
particular the present invention discloses an intuitive system for digital image acquisition, 
organization, manipulation, and publication. 

10 BACKGROUND OF THE INVENTION 

*P With traditional amateur film photography, the process often ends after a 

fli 

yQ film roll has been developed and returned to the photographer. The photographer obtains 

a p the developed photographs in an envelope and reviews the photographs with friends and 

□ 15 family. Some energetic amateur photographers select their favorite photographs and 

□ place those photographs into a photo album. However, most amateur photographs simply 
jSLjj end up in a shoebox at the back of the closet. Occasionally, the shoebox is retrieved such 
1 y that friends and family may view the photographs. 

20 With the advent of digital photography, the possibility for organizing and 

publishing amateur photographs has greatly increased. Amateur photographs can be 
published on the world-wide web (WWW) to allow anyone on the Internet to view the 
photographs. Image manipulation programs can be used to manipulate digital 
photographs in order to correct flaws, modify colors, etc. However, the potential of 

25 amateur digital photography largely remains unrealized. Most digital photographs are 



APLE.P0018 



just stashed away on hard disk drives are burned onto Compact Disc Read-Only 
Memories (CD-ROMs) and only occasionally viewed. A few digital photographs are 
emailed to friends and family. 

To better realize the great potential of amateur digital photography, it is 
clear that better tools are needed for amateur digital photography enthusiasts. These tools 
should be highly intuitive yet provide powerful features for the acquisition, organization, 
manipulation, and publication of digital images. 
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SUMMARY OF THE INVENTION 



To better realize the great potential of amateur digital photography, the 
present invention introduces an integrated system for the acquisition, organization, 
manipulation, and publication of digital images by amateur digital photography 
enthusiasts. The system of the present invention first acquires images from a number of 
different image sources. Images acquired in the same image importing session are 
marked as coming from the same conceptual film roll. Next, a user is empowered to 
organize and manipulate the acquired images. The images may be organized by tagging 
the images with informative keywords and grouping images together into conceptual 
photo albums. The images may be manipulated by rotating, cropping, and removing red- 
eye. Finally, the system of the present invention provides simple intuitive image publish 
systems. A selected group of images may be published in a photobook, published onto 
the World Wide Web, or published as individual image prints with minimal computer 
knowledge on the part of the user. 

Other objects, features, and advantages of present invention will be 
apparent from the company drawings and from the following detailed description. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



The objects, features, and advantages of the present invention will be 
apparent to one skilled in the art in view of the following detailed description in which: 

5 

Figure 1 illustrates a flow diagram that describes the over all flow from 
image acquisition to image publication using the system of the present invention. 

Figure 2 illustrates a block diagram of a computer system that may be 
10 used to implement the teachings of the present invention. 

Figure 3 illustrates one embodiment of the main display screen of the 
image organization and publishing system of the present invention. 

15 Figure 4 illustrates a view of the main display screen with film roll 

dividers appearing between different groups of images acquired into the image 
organization and publishing system at different times. 

Figure 5a illustrates the landscape aspect ratio for an image. 

20 

Figure 5b illustrates the portrait aspect ratio for an image. 

Figure 5c illustrates a dotted line square box that ensures either a 
landscape or a portrait image will fit within its borders. 

25 
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Figure 5d illustrates an example of image review window that displays 
images using the dotted line square box of Figure 5c. 

Figure 6 illustrates an example of a main image review window that 
displays thumbnail images with view packing system of the present invention. 

Figure 7 illustrates a flow diagram describing the flow of producing a 
photobook using the system of the present invention. 

Figure 8a illustrates a style sheet for a portrait image page. 

Figure 8b illustrates a style sheet for a landscape image page. 

Figure 8c illustrates a style sheet for a dual portrait image page. 

Figure 8d illustrates a style sheet for a dual landscape image page. 

Figure 8e illustrates a style sheet for a image page with a landscape image 
and a portrait image. 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 



A method and apparatus for the acquisition, organization, manipulation, 
and publication of digital images is disclosed. In the following description, for purposes 
5 of explanation, specific nomenclature is set forth to provide a thorough understanding of 
the present invention. However, it will be apparent to one skilled in the art that these 
specific details are not required in order to practice the present invention. For example, 
the present invention has been described with reference to publishing images in a 
photobook, on a World Wide Web page, and as individual photographic prints. However, 
10 the system of the present invention can be used to publish images in other formats such as 
p postcards, magazines, and email messages. 

[j~ Image Manipulation & Publishing System Overview 

;U 15 To better realize the great potential of amateur digital photography, the 

JI! present invention introduces an integrated system for the acquisition, organization, 

|*J manipulation, and publication of digital images by amateur digital photography 

CU enthusiasts. Figure 1 illustrates a flow diagram that describes the over all flow from 

image acquisition to image publication using the image organization and publishing 
20 system of the present invention. 

Image Processing Flow 

Referring to step 120 of Figure 1, the first task is to capture images that 
will be organized, manipulated, and published. The system of the present invention does 
25 not deal with the image capture stage since that stage is the purview of digital cameras 
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and color scanner systems. Note that the images may be captured in any original source 
format (chemical film photography or electronic digital photography) as long as a digital 
version of the original source image is available. 



5 The image organization and publishing system of the present invention 

begins at step 130 where image acquisition is performed. The goal of the image 
acquisition step is to get the images onto a computer system that will be used to organize, 
manipulate, and publish the images. The system of the present invention allows digital 
images to be acquired from various different sources such as digital cameras, CD-ROMS, 
10 and across computer networks. The image acquisition system of the present invention 
records certain information to help organize the images. 



jjj After acquiring the captured images on the computer system, the acquired 

J images are organized and manipulated at step 140. The image organization and 

L 15 publishing system of the present invention provides a number of intuitive yet powerful 
[I tools for organizing and manipulating the acquired images. 

~- 

W At step 150, the user of the image organization and publishing system of 

the present invention selects a set of desired images for publication. The user further 
20 selects a particular image publication system to use since the system of the present 

invention uses an open architecture that provides more than one image publication system 
and allows additional publications systems to be added later. 

Finally, at step 160 in Figure 1, the image organization and publishing 
25 system of the present invention publishes the selected images using the selected 
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publication system. Step 160 is illustrated in a dotted box because not all of the work in 
step 160 will be performed by the image organization and publishing system of the 
present invention. For example, for certain publication systems, the image organization 
and publishing system will send processed image data to another outside system that will 
5 provide an end result such as a published photo book. 

Image Processing Computer System 

Figure 2 illustrates an exemplary embodiment of a computer system 200 
that may be used to implement the teachings of the present invention. Computer system 

10 200 may be a typical personal computer system with a processor 210, a memory 220, and 
a bus controller 230 for controlling a system bus 235. Various different Input/Output 
devices may be coupled to the system bus 235 to allow the computer system 200 to 
interact with the outside world. For example, computer system 200 includes the 
input/output devices of a hard disk drive (HDD) 269, a display device 261 (such as a CRT 

15 or flat panel display), a keyboard 251, a cursor control device 253 (such as a computer 
mouse or track pad), a Compact Disc Read-Only-Memory (CD-ROM) Drive 280, and 
network interface 291 for coupling to a network such as local area network (LAN) 295. 

The image organization and publishing system 225 of the present 
20 invention resides in the memory 220 of computer system 200 as a set of computer 

instructions. The image organization and publishing system may also reside as a program 
on a media device such as CD-ROM 281 used with CD-ROM drive 280. Alternatively, 
the image organization and publishing system may be obtained across a network such as 
the global Internet 299 accessible through network interface 291. 
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Image Acquisition 



10 



Referring back to step 130 in Figure 1, the first task of the image 
organization and publishing system of the present invention is to acquire digital images to 
process. Referring to Figure 2, the acquired images are normally stored onto a mass 
storage device such as hard disk drive (HDD) 269. In one embodiment of the present 
invention, the image organization and publishing system 225 may acquire digital images 
from a number of different sources. 



Digital Camera Image Acquisition 

In a preferred embodiment of the present invention, the image organization 
and publishing system of the present invention includes driver software necessary to 
automatically identify and interoperate with several different types of digital cameras that 
15 may be coupled to the host computer system. 

For example, Figure 2 illustrates a digital camera system 247 coupled to 
serial bus 245 controlled by serial bus interface 240 of the computer system 200. Serial 
bus 245 may be a Universal Serial Bus (USB), an IEEE 1394 Firewire serial bus, or any 
20 other type of serial bus. 

In a preferred embodiment, image organization and publishing system 225 
is able to automatically recognize when digital camera system 247 is coupled to computer 
system 200 such that the digital images in digital camera system 247 can be acquired. In 
25 one embodiment, the image organization and publishing system 225 would ask the user if 
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he or she would like to delete the transferred images off of digital camera system 247 
such that the same digital images would not be transferred again at a later time. 

Other Image Acquisition Means 
5 The image organization and publishing system 225 can also acquire 

images in a number of other manners. For example, film photographs may be placed onto 
a floppy disc (not shown) or onto CD-ROM 281 by the film developer at the request of 
the film photographer. Such photos may be acquire by image organization and publishing 
system 225 by having the user place the floppy disc (not shown) or CD-ROM 281 into the 
10 appropriate drive such as floppy disc drive (not show) or CD-ROM drive 280. 

j! Images may also be obtained through a computer network interface 291 of 

m 

yg computer system 200. For example, a user may obtain images from a World Wide Web 

J- (WWW) site on the global Internet 299. Finally, images may also be obtained using a 

f i 15 scanner (not shown) coupled to computer system 200 as is well known in the art. 

:??-!; 

% Image Set Grouping 

m When the image organization and publishing system 225 acquires a set of 

images, the image organization and publishing system 225 identifies all the images in 
20 some manner to indicate that the set of images are related. This objective may be 

achieved by marking all the images in the set with a unique set identifier. In this manner, 
a set of images that are likely to be related will be kept together. 

For example, when a user couples digital camera system 247 to computer 
25 system 200, the image organization and publishing system 225 may recognize digital 
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camera system 247 and inform the user that images may be transferred from digital 
camera system 247. If the user transfers images from digital camera system 247 onto 
computer system 200, the image organization and publishing system 225 indicates that all 
the images transferring during that session are related. As previously set forth, this may 
5 be achieved by marking all the images in the set with a unique set identifier. 

Alternatively, the transferred images may all be given the same file creation date and time 
such that any program accessing those transferred images could infer that the images are 
related since the images were all created at the same date and time. 



10 



Image Organization and Manipulation 



4j Once images have been acquired into the computer system 200 used by the 

4S image organization and publishing system 225, the image organization and publishing 

□ 15 system 225 displays the images to the user such that the user may organize, manipulate, 
and publish the images. Figure 3 illustrates one embodiment of the main display screen 
300 of the image organization and publishing system 225. 

Referring to Figure 3, a main image review window 310 allows the user to 
20 view all the images that have been acquired in the image organization and publishing 

system. The available digital images are displayed as thumbnail summary images within 
main image review window 310. 



The size of the thumbnail summary images displayed within main image 
25 review window 310 may be varied using image size slider 313. By setting the image size 
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slider 313 to make the thumbnail images very small, a large number of images may be 
viewed simultaneously within main image review window 310. Conversely, by setting 
the image size slider to make the images large, fewer images may be view simultaneously 
but more detail can be seen in each image displayed. The user may scroll through the 
5 various images using scroll bar 315. The main image review window 310 acts as the 
"shoebox" for storing all the images available to the user. 

Film Roll Concept 

As set forth in the previous section, images that are acquired into the 
image organization and publishing system at the same time are grouped together into a 
related set. In one embodiment, such groupings are conceptually viewed as different 
"film rolls". 

The main image review window 310 can be used to display such film roll 
groupings. Figure 4 illustrates a view of the main display screen with a "film roll" 
checkbox 453 checked such that film roll dividers appear between different groups of 
images acquired into the image organization and publishing system at different times. 
Specifically, the first two images are from a first "film roll" (image acquisition time), the 
next five images are from a second film roll, the two images are from a third film roll, 
and so on. 

View Packing 

The images displayed in the main image review window are presented in 
an aesthetically pleasing manner that optimizes the use of space. Specifically, the present 
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invention introduces a "view packing" system that is analogous to proportional spacing in 
written documents. 

Images generally come in two different aspect formats: landscape and 
portrait. Figure 5a illustrates the landscape format named since that format is generally 
used when photographing landscape images. Figure 5b illustrates the portrait format that 
is generally used when taking a portrait photograph of human. 

One method of displaying both landscape and portrait images together in 
an organized manner is to reserve a space for either shaped image. Figure 5c illustrates a 
dotted line square box that ensures either a landscape or a portrait image will fit within its 
borders. Figure 5d illustrates an example of image review window that displays images 
using the dotted line square box of Figure 5c. 

To more efficiently use display screen real estate, the view packing system 
of the present invention sequentially examines the ordered list of images and, for each 
image row, takes as many images as will fit on a row with an allowance for a minimum 
space between images. If there are a large number of consecutive narrow portrait images, 
then a large number of images will be displayed on a particular row. Alternatively, if 
there are a large number of consecutive wider landscape images, then fewer images will 
be displayed on a row. 

Figure 6 illustrates an example of a main image review window 600 that 
displays thumbnail images with view packing. Referring to the first row 610 of main 
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image review window 600, only four landscape images could fit in the row. The extra 
horizontal space for row 610 was uniformly placed between the landscape images. 

In the second row, row 620, six narrow portrait images are presented. The 
5 narrow portrait images thus allow more images to be displayed on a particular row. 
However, note that row 620 is taller than row 610 that only contains landscape images. 

Row 630 contains two landscape images and three portrait images. Due 
the presence of the tall portrait images, the mixed row 630 takes as much vertical space as 
portrait only row 620. However, the narrow portrait images do allow for some efficiency 
in the horizontal direction such that five images are displayed in row 630. Note that in 
mixed row 630, all the images (both portrait and landscape) are centered in the vertical 
direction. 

q 15 By comparing the view packing system of Figure 6 to the prior art system 

U of Figure 5, one can easily see the display screen real estate efficiencies of the view 

JLj packing system of the present invention. 

Image Keyword Tagging 
20 To help organize the images in the user's "shoebox", the present invention 

provides an intuitive keyword tagging system. Referring back to Figure 3, the main 
display screen 300 includes a keyword tagging bar 340 for tagging images in the main 
image review window 310. 
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To tag a particular image or a set of images, a user simply selects the 
desired image or images in the main image review window 310 and then clicks on one of 
the keywords in the keyword tagging bar 340. To display all the keyword tags associated 
with the images, the user may click the "keywords" checkbox 352 such that keywords 
5 associated with an image are display below that image in the main image review window 
310. Similarly, the user may display titles associated with each image by clicking on the 
titles checkbox 351. 



The keyword tagging bar 340 initially contains a number of common 
10 keywords that are relevant to photographs. For example, the keywords "Family", 

"Vacation", and "Holiday" allow a person to tag images as being related to their family 
members, their vacations, or holiday celebrations, respectively. However, empty spaces 
are available on the keyword tagging bar 340 to allow the user to create their own 
personal keywords. In one embodiment, the user may apply a "keyword filter" such that 
15 the main image review window 310 only displays images containing a particular keyword 
or set of keywords. 

Two special keywords, "none" 341 and "0" 342, are also presented on the 
keyword tagging bar 340. The "none" 341 keyword specifies that an image should have 
20 no keyword tags. Thus, the "none" 341 keyword can be used to clear existing keywords 
on an image. 



The "0" 342 (check) keyword is a special keyword that is displayed on 
the front of the thumbnail version of an image in the main image review window 310. 
25 The "0" 342 keyword is used to provide the user with a means for selecting a particular 
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image from a group of similar images. For example, the top row of images in the main 
image review window 310 may contain four images of the same landscape. The user may 
carefully review all four images and select the third as the best image of the four images 
of the same landscape. To remember that selection, the user may mark the third image 
5 using the "0" 342 keyword button. Thus, the user will be reminded of that choice as 
illustrated by the 0 icon on the third image of the first row. 

Photo Albums 

To further organize the images, a user of the image organization and 
10 publishing system allows the user to create conceptual "photo albums." A photo album is 
p a specific ordered list of images from the overall shoebox containing all the images. Each 

,j= photo album is given a descriptive title. 

J Referring again to Figure 3, the photo album list area 330 contains a list 

!L 15 of photo albums. In Figure 3, there are five photo albums titled "Mexico Trip", 
g "Christmas 200 1 ", "Yosemite Park", "Santa Cruz", and "Billy." In one embodiment, a 

J user may view the images in a particular photo album by selecting the name of the photo 

!U album such that only the images from that photo album are display in the main image 

review window 310. Note that images in a photo album are in a specific order and that 
20 order is reflect in the main image review window 310. The user may rearrange the order 

of the images by dragging and dropping the images within main image review window 

310. 

To create a new photo album, the user clicks on new album button 321 
25 that is marked with a "+". The user provides a descriptive name for the new photo album 
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and then may begin dragging images from the shoebox and dropping the images onto the 
name of the new photo album. Additional information may be obtained about a particular 
photo album by selecting the photo album name and then clicking the information button 
323. 

5 

To generate a quick slide show of the images in a photo album, the user 
may select a particular photo album name and then click the slide show "play" button 
322. The image organization and publishing system will then sequentially display the 
images in full screen form on the display system. A music track may be selected to 
10 accompany the slide show. 

The slide show "play" button 322 may also be used to create a slide show 
of any group of images. Specifically, the user may select any group of images in the main 
image review window 310 and then click the slide show "play" button 322 to create a 
15 slide show of the selected images. 

Image Editing 

The image organization and publishing system of the present invention 
provides a set of features for editing the acquired images. Most of the image editing tools 
20 are available in a separate image editing mode. However, referring to Figure 3, an image 
in the main image review window 310 may be rotated by selecting the image and then 
clicking the rotate button 324 that will rotate the image ninety degrees clockwise. Images 
may be rotated counter-clockwise by holding down the keyboard shift key while clicking 
the rotate button 324. 

25 
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In an image-editing mode, the present invention provides a set of intuitive 
image editing tools for editing images. One editing tool is a "red-eye" removal tool To 
use the red-eye tool, a user should highlight the area of an image containing the red eyes 
and then clicking the red-eye tool The user may also just click the red-eye tool without 
5 selecting an area of the image but better results will be provided when only a certain area 
of the screen is selected. 

In order to prepare images for enlargement, the image-editing mode 
provides an image-cropping tool. In a preferred embodiment, the image-cropping tool 
10 allows the user to select the most common aspect ratios used for creating reprints. 
O Specifically, the image-editing mode provides a cropping tool that can be locked in 3x5, 

£ 4x6, 5x7, and 8x10 aspect ratios. In this manner, the user can select exactly how an 

enlargement will appear as opposed to the current traditional photography system wherein 
^ the film developer crops the image as needed to obtain the aspect ration of the selected 

p 15 reprint size. 

O 

JLj In one embodiment, the image-editing mode of the present invention 

' y creates a new image file containing all the edits applied by the user. However, the 

original file of the unedited image is still kept unaltered. In this manner, the user may go 
20 back and retrieve the original image if it is later needed. 



DHJ 



-19- 



APLE.P0018 



Image Publication 



After a user has organized and manipulated a set of images, the user may 
then wish to publish the organized and manipulated a set of images. The present 
invention provides an expandable system for publishing images. Specifically, the image 
organization and publishing system uses individual image-publishing modules of code to 
provide image-publishing services. 

Three image-publishing modules are included in an initial embodiment of 
the present invention. Specifically, one embodiment includes a Photobook publication 
module, a World Wide Web Homepage publication module, and a photograph reprint 
publication module. 

Photobook Publication 

The photobook publication module allows a user to quickly and easily 
create an aesthetically pleasing photobook. The creation of a photobook will be described 
with reference to the flow diagram in Figure 7 and the style sheets in Figures 8a to 8e. 

Referring to Figure 7, the user must first create and select a photo album 
as set forth in step 720. The creation of a photo album is described in the previous 
sections. The user the selects the desired photo album and enters the photobook 
publication module as set forth in step 730. 

Next, in step 740, the user must select a photobook style. The photobook 
publication module contains a number of different photobook styles that may be selected. 
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The photobook styles describe the overall appearance of the photobook that can be 
created. 

Each photobook style consists of a series of different style sheets that 
describe how each page of the photobook will appear. Figures 8a and 8b illustrates two 
possible style sheets for a single image page in a photobook. Specifically, Figure 8a 
illustrates a style sheet for a portrait image page wherein the solid line area is the location 
for the image and the dotted line area is for a caption associated with the image. 
Similarly, Figure 8b illustrates a style sheet for a landscape image page. 

Figures 8c, 8d, and 8e illustrate possible style sheets for a dual image 
page in a photobook. The style sheet of Figure 8c is for two portrait images, the style 
sheet of Figure 8d is for two landscape images, and the style sheet of Figure 8e is for a 
page with one landscape image and one portrait image. 

The photobook publication module fills in style sheet pages in the 
template photobook style until all the images in the photo album are exhausted. The 
photobook publication module automatically selects the proper style sheet depending on 
the image aspect ratio (portrait or landscape). If the number of images in the photo album 
does not match up perfectly, one embodiment of the present invention leaves the final 
page incomplete. In an alternate embodiment, the photobook publication module selects 
a style sheet with the exact number images needed for the last page. 

Referring again to Figure 7, the user may then manipulate the 
automatically formatted photobook at step 750. Specifically, the user should fill in the 
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captions for the images. The user may reorder the pages in a photobook. The user may 
also reorder images or delete images from the photobook. In one embodiment, when the 
user reorders pages, reorders images, or deletes images in a photobook then the 
photobook publication module propagates the changes back to the photo album used to 
5 create the photobook. 



In one embodiment, there is a "lock page" checkbox for each page in the 
photobook layout. The lock page checkbox can be used to prevent other changes made to 
the photobook from affecting the locked page. For example, if the user locks a particular 
10 photobook page and then deletes an earlier image, the images in the locked page will not 
be split up in order to fill in for the missing image. Instead, one embodiment of the 
present invention selects a style sheet with one less image needed for the page before the 
locked page. In an alternative embodiment, an image after the locked page will be moved 
in front of the locked page. 



U 15 



Finally, when the user is satisfied with the photobook layout, the user 
requests a final photobook formatting at step 760. In one embodiment, the photobook 
publication module processes the arranged photobook to create a Portable Document 
Format (PDF) file. Details on the Portable Document Format can be found from Adobe 
20 Systems, Inc. of San Jose, California. 

After creating the final formatting of the photobook, the user may publish 
the photobook at step 770 by printing the photobook on a local printer. However, to 
obtain a hardbound photobook, the photobook publishing module allows the user to 
25 submit the formatted photobook directly to a custom book publisher across the Internet. 
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Note that this system uses the locally available computing power to create the photobook 
such that a difficult editing session across a computer network is not necessary. 

Home Page Publication 

A user may publish their images on the World Wide Web using a World 
Wide Web Homepage publication module. The World Wide Web Homepage publication 
module operates in a very similar manner as the previously described photobook 
publication module. 

Specifically, the user first selects a photo album and a template World 
Wide Web Homepage style. The World Wide Web Homepage publication module then 
generates a proposed World Wide Web Homepage. The user may then edit the proposed 
World Wide Web Homepage. When the user completes the editing of the proposed 
World Wide Web Homepage, the user may then submit the created World Wide Web 
Homepage to a World Wide Web hosting service. 

Individual Print Publication 

Finally, the user may publish images as individual prints. To publish a set 
of prints, a user selects a group of images from the main image review window 310 or 
selects a photo album from the photo album area 330 of Figure 3 and then requests 
reprints. The photograph reprint publication module then displays an electronic order 
form to the user with the selected images. The user may then select the quantity and size 
of desired for each of the images. A "quick order" selection may be used to indicate a 
desired quantity for all the selected images. In one embodiment, the quick order selection 
also allows the user to select a desired size. 
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The foregoing has described an image organization and publishing system. 
It is contemplated that changes and modifications may be made by one of ordinary skill in 
the art, to the materials and arrangements of elements of the present invention without 
departing from the scope of the invention. 
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